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ABSTRACT 



Solano Community College (California) developed 
interdisciplinary, team- taught classes. The program included three classroom 
components administered in each of the six semesters to both 
program- supported integrated classes and to content area control class 
sections: a standardized reading test, an essay writing test, and a student 
survey. Subjects were 147 students enrolled in English 1, history, biology, 
social science, political science, and nutrition classes. In addition, 
summative fill and retention data were gathered to compare the program 
sections with all other sections of the same course, and a follow-up study 
surveyed students who completed the program's integrated classes. Results 
indicated that the essay writing evaluation produced some evidence for the 
success of the pedagogical method, but the reading evaluation did not reveal 
substantive changes in students' abilities. Although no formal mechanism was 
used to evaluate it, a "tremendously positive" collegiality developed among 
the faculty involved in the program. (Contains four tables of data; 
appendixes contain pre- and post -essay topics, essay scoring rubrics, sample 
student survey forms, sample follow-up survey forms, and an evaluation 
activities summary.) (RS) 
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The evaluation of this FIPSE Project has included three classroom components administered in 
each of the six semesters to both the Project-supported integrated classes and to content area control 
class sections: a standardized reading test, an essay writing test, and a student survey. In addition, 
summative fill and retention data have been gathered to compare the Project sections with all other 
sections of the same course, and a follow-up study has surveyed students who have completed the 
Project integrated classes. 

Classroom Evaluation 

This project has supported the teaching of two integrated courses in each of six semesters. 
Each pair of instructors has repeated the integrated course in a subsequent semester; during the 
second offering the courses have been evaluated by comparison with one or more control sections of 
the same content area course. I wish to thank the following instructors for participating in this 
evaluation: 



SEMESTER 


COURSES 


INSTRUCTORS 


Fall 1992 


History 17 

History 17 + English 1 


Control Sections: Katie Graham 
Van Forsyth 
Integrated Class: Diane White 

Susan Messina 


Spring 1 993 


Biology 15 

Biology 1 5 + English 1 


Control Sections: Jim Mills 
Integrated Class: John Nogue 
Nan Wishner 


Fall 1993 


Social Science 22 

Social Science 22 + English 1 


Control Section: Anne Bevilacqua 
Integrated Class: Lucy DuPertuis 
Lue Cobene 


Spring 1 994 


History 5 

History 5 + English 1 


Control Section: Jerry Korte 
Integrated Class: Richard Bunch 
Laurie Duesing 


Fall 1994 


Political Science 1 

Political Science 1 + English 1 


Control Section: David Pyle 

Integrated Class: Jean Baxter 
Sarah Phelan 


Spring 1995 


Nutrition 10 

Nutrition 10 + English 1 


Control Section: Carol Bishop 

Integrated Class: Nancy DuPuy 
Susan Messina 



All of these instructors modified their instruction in the evaluation semester to accommodate the 
standardized reading tests, the in-class surveys, and the teaching of curriculum units on which the pre 
and post essay tests were based; we are grateful to them for their contributions to the success of this 
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evaluation. In addition, thanks are due to the faculty from all of these disciplines who joined the essay 
scoring teams: Richard Green, Jim DeKloe, Paul Hardy, Bill Hughes, Judy Kabica, and Sharyn Stever. 

Evaluation Methods 

In all of the six semesters, the classroom evaluation consisted of three pre and post semester 
administrations: 

1 . Degrees of Reading Power standardized critical reading test (Forms R-2 and S-2) 

2. Locally constructed essay writing test based on a short curriculum unit 

3. Short open-ended student survey (several different forms). 

Parlic i pa ting_Studenls. The community college students participating in this evaluation 
consisted of those enrolled in the Project-supported integrated classes listed above (an aggregate of 
147 enrolled at the beginning of the six semesters) and the students enrolled in the content area 
control classes. The integrated classes were limited to 30 students which is the English department 
maximum for English 1, even though this meant a reduced number of students from the content area 
perspective, where lecture classes typically enroll 50 students or more. The History 17 cohort had two 
control sections, and the Biology 15 lecture had the equivalent of four control sections all taught in a 
single lecture hall. For the four remaining cohorts, there was just one control class, in each case a 
lecture class that began with about 50 students per section. 

Although all students in the control class did the tests and surveys regardless of their English 
status, only those students who were simultaneously taking the same composition course, English 1, 
as the integrated class students, were included in the data reported for the reading and essay tests. 
Thus, comparisons made in the reading and writing components of this evaluation are based on 
similarly qualified students (there are specfic prerequisites for English 1) taking the same two courses 
in English composition and a specific content area. 

As can be seen in Table 1, the number of such students completing both English and the 
content area class was very small in some control sections. The evaluation was designed based on a 
previous classroom research project with a History 17 class in which the number of students taking 
English 1 in 2 History 17 sections equalled the number in an integrated class, as was the case in the 
History 17 cohort in this project, but we did not find an equivalent number of students taking English 1 
simultaneously in other content area classes. Thus, our exact control students (those taking the same 
composition course simultaneously but in separate classes) did not equal the number in the Project 
classes in any of the last 3 semesters of the Project. 

Furthermore, some students (both exact control and integrated) took only one of the two 
reading or writing tests, but not both; thus the number of students listed in Table 1 as earning a final 
grade in both integrated and control classes is not the same as the number who took both reading tests 
or both writing tests. (See the specific number of students in both groups for each component in Table 
1 .) Only students who took both reading tests are included in the DRP data reported in Table 1, and 
only students who took both essay writing tests are reported in the essay data in Table 1 . And only 
students who got a final grade that was not a W (withdrawal) in both English 1 and the content area 
course were included; this meant some control students who completed the content area course but 
dropped English 1 were dropped from the evaluation data base. Thus, because of the small number of 
control students in some semesters, our aggregated data are more valid than those for any individual 
class, and tests of statistical significance are reported only for the aggregate data. 

The in-class surveys were also problematic: not all students completed a survey, and 
anonymity prevented any match-up of pre and post student opinions. In addition, because there is the 
mix of English levels in the survey data, it cannot be said to represent students who are all comparable 
to the students in the integrated project classes. In general, it is fair to say that students in the 
integrated classes wrote more and more substantive comments on the surveys than did the students in 
the control classes. 
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B e ading^valua t io n - M alarial s a nd Proc edure^ The students' critical reading skills were assessed 
with the Degrees of Reading Power (DRP) tests in forms R-2 and S-2. In this test, students are asked 
to read passages that are approximately one full page in length and choose appropriate main idea 
statements for paragraphs, draw inferences and generalizations, and make comparisons; they are not 
asked factual detail questions. Scores are reported on a DRP constructed scale of reading difficulty. 
TASA, the publisher of the DRP, also has a software program which measures the reading difficulty of 
text passages; passages from reading assignments in all six semesters were measured by this means, 
so students could compare their personal test scores with the level of the assigned readings. Topics in 
the DRP passages are of an academic nature and cover a wide range of curriculum areas, including 
business, history, science and literature. These tests are untimed so that slow, but good readers- 
especially ESL students-are not at a disadvantage. Arrangements were made for students who did not 
complete the test within the class time to do so after class. The DRP tests were given in 50-minute 
class periods in the first 3 weeks and last 3 weeks of the semester; the experimental and control 
classes took the tests within one or two days of each other so that there is no substantive time 
difference between the two evaluation groups. 

Reading Evaluation-Results. As can be seen in Table 1, the Project students scored 
statistically significantly higher on both the DRP pre-test and post-test. The significant difference in 
reading gain score is due more to a decline in mean control group scores than to increase in mean 
integrated class group scores, which increased less than one point. Based on my observation of 
control class students taking this test and on comments made by control class instructors, I suspect 
that control class students took these tests less seriously than they did the writing tests because the 
score earned on the DRP did not affect their grade, whereas the score earned on the essay test did. 
Although standardized tests have the advantange of comparison with other students across the 
country, students may not be motivated to do well on these tests. 

One issue of concern is that despite the semester of reading and writing instruction in English 
1, these students-especially the students in Project classes, which emphasized substantial reading-did 
not show more improvement. A second, related issue is the instructors' concern that the mean scores 
are well below the level of difficulty of typical reading assignments in all of the content area courses 
included in the Project. Using the DRP software, passages from textbooks and all of the articles 
chosen for the curriculum units on which the essay tests were based were analyzed for difficulty level; 
most fell into the 70-78 range on the DRP scale, with only a few in the high 60 f s. Thus, the average 
student reading skill level in the Project classes was 10-15 points below the assigned readings. This 
gap may explain why neither group scored at an average level on the synthesis subscore in the writing 
test. Students' ability to comprehend and synthesize multiple readings appears to be not yet 
adequately developed. 



The essay writing tests were based on curriculum 
units taught in both classes for 1 to 3 weeks on topics that were normally covered in the semester in 
each course. Along with the reading test and survey, this instruction and the essay writing test also 
occurred in the first three and last three weeks of the semester. Six faculty comprised the essay 
scoring team: the English and content area instructors from the integrated class and the control class, 
an instructor who has taught the specific course in both English and the content areas but who was 
not teaching a FIPSE Project supported class, and I as English 1 instructor and evaluation director. This 
team met several weeks before the evaluation semester began and planned administration dates and 
the curriculum unit topics and readings. It took considerable discussion to get the content area 
instructors of both classes to agree on exactly what would be taught, but everyone involved-especially 
part-time instructors-said it gaive them a better idea of what other people thought was important to 
teach. An outline of topics to be covered was developed and followed in each class, so all students 
were exposed to the same content information. Three to six readings were chosen so that all students 
had the same assigned reading. One reading in each curriculum unit was left undiscussed in class, so 
we could see how students could handle reading which had not been covered in class. 
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Table 1. Mean Scores for Reading Tests (Degrees of Reading Power), Essay Totals and Subscores, Reading and Writing Gains, and Final Course Grades 
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What was different, of course, was the method of instruction. Most of the control class 
instructors presented the material in typical lecture style, although small groups were used in the 
nutrition control class to discuss the readings after the lecture. The FIPSE Project classes have 
emphasized small group discussion of the readings, so that was the dominant method of instruction in 
the integrated classes. 

After the curriculum unit had been taught and the readings discussed, the scoring team of six 
faculty met to compose the specific question on which the students would write their essays. 

Although there was considerable time pressure, the groups met before the next day's essay writing. 
Writing the question in this manner assured that none of the faculty could teach to the question, which 
•s very hard to avoid when it is known while instruction and discussion are going on. 

Two members of the scoring team (English instructors Messina and Stever) worked on all six 
essay teams, so there was continuity in the types of questions created for each semester. All but one 
of the faculty teams were adamant that the essay topics be analytical in nature, not just information 
repetition, and the same groups wanted students to be able to read a variety of sources, pull out 
important points to use in supporting the claims made in their essays, and be able to apply important 
concepts to new situations. All teams wanted students to create a coherent framework within which 
to present their information and claims. Faculty were pleased to hear that other instructors expected a 
similar quality of thinking, though we all recognized that students' abilities often fall far short of our 
expectations. 

To score the essays, an analytic rubric was developed by the scoring team of six faculty. 

Since they were the first group, the History 17 group set a pattern that included content, synthesis and 
analysis, organization, and mechanics as the primary criteria for scoring, with more points to be 
allocated to content, and fewest to mechanics. The next group, Biology 15, spent some time starting 
“from scratch” just to see what they thought were the important criteria for evaluating writing, but they 
came up with very similar criteria. Thus, after 9 separate faculty members from disciplines as 
divergent as English, social science, and science agreed that the same criteria were important, we 
settled on those five items for analyzing student writing; the only change was to separate synthesis 
and analysis, which had been in one category in the History scoring. Subsequently, the History papers 
were rescored so that those scores would fit into the cumulative data base (see Table 1). The rubrics 
were changed slightly as each scoring group defined exactly what they mean by “content,” “synthesis,” 
etc., and they were changed to reflect the different topics at the beginning of the semester and the 
end of the semester. Essay topics from all six semesters are included in Appendix A, and rubrics 
appear in Appendix B. 

The essay scoring was done in one six-hour session or two three-hour sessions on adjacent 
days. Robert Simas removed the students' and instructors' names so that papers were identified only 
by a code number. Susan Messina led each six member scoring team in a discussion of sets of five 
papers to establish norms for scoring so that we were all evaluating the essays similarly. Each essay 
was read by one English instructor and one content area instructor. Agreement in scoring was defined 
as both readers' scores falling within a 20% spread (8 points on our 40 point scale). If the two readers 
did not agree, a third-and occasionally, a fourth-reader scored the essay until agreement was reached 
between at least one English and one content area instructor. (We were unable to reach agreement on 
two papers at most of the more than 80 in the typical scoring session.) Score sheets were then 
removed by Robert Simas and scores entered into the data base. Name sheets were reattached and 
essays returned to the instructors for their own grading. All instructors counted these essays as a 
significant part of the students' semester grades. 

W.riting_£valijatioii-RfisuJts. Aggregate group means for the total essay scores and the five 
subscores appear in Table 1 . Although the FIPSE Project integrated classes scored higher in all 
categories on Essay 1, that difference is statistically significant (using t-test) only on organization and 
mechanics subscores. This difference may indicate that students who were already somewhat more 
skilled in these two aspects of writing were more likely to choose the integrated class to further 
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develop their writing competence. However, on the writing score as a whole, there was no significant 
difference between the groups on Essay 1 (pre-test) scores; thus we can assume that both groups 
were about equal in initial writing abilities. However, it is interesting to note that both groups were 
below the score of 20, which was regarded as “average.” 

The FIPSE integrated class group did score statistically significantly higher on the post-test, 
Essay 2, on the overall score and on all subscores except synthesis. Each of these scores, except 
synthesis, is above the midpoint score defined as “average,” while the overall score and the subscores 
of the control classes remain below “average,” on Essay 2. Although the writing gain score is not 
statistically significant, its trend is to show greater gain for the integrated class students. These are 
rewarding results for those of us who have worked very hard to create a meaningful context in which 
to embed writing instruction. 

Another way of analyzing the essay writing results is to consider how many students' essays 
fell into four qualitative categories: Weak, Low, Good, Excellent. Table 2 displays the number and 
percentage of students falling into each category on Essay 1 and Essay 2. 

There is no statistically significant difference in the percentages of students in each category 
on either Essay 1 or Essay 2 total scores. However, one goal of the FIPSE supported writing 
instruction is to try to improve students' ability to synthesize and analyze, and there does seem to be 
some measurable improvement in those subcategories. On the Essay 2 synthesis subscore, there is a 
statistically significant higher percentage of students from the integrated classes who have moved into 
the Excellent category (Z score = 1 .979), while the percentage of control class students in that 
category remains the same. 

On the analysis subscore, the difference in percentage of integrated class students in the 
Excellent category is approaching statistical significance, and the percentage of integrated class 
students in the Weak category is significantly smaller than the percentage of control students (Z score 
= 2.255). (The z score is a non-parametric test for significant difference between two percentages.) 

Discussion_of_Heading_andJA/riting.JEvalua±iQrL The results of the essay writing evaluation show that 
there are some statistically significant differences between the writing performance of students in the 
FIPSE- supported integrated classes compared with students taking the same courses separately. 

Given the faculty desire to have students analyze and synthesize the content area ideas and 
information, this result is encouraging. It supports an instructional methodology which gets the 
students actively engaged in reading, thinking, and discussing the course topics, as opposed to the 
more passive learner role students usually adopt in a lecture class, where their main role is to listen. 

One concern both faculty and FIPSE staff had before we began this project was whether 
students would learn the content of the courses just as well in the integrated environment. The course 
grade results seem to show that they learn content even better than their counterparts in separate 
classes. While the mean English grade is the same, the mean content grade earned by integrated class 
students is significantly higher (See Table 1). One additional issue addressed by the course grade data 
is that there is little evidence in these courses of the grade inflation so commonly decried in other 
courses. This could be due to the substantive amount of reading and writing that were a necessary 
part of the two curriculum units. 

Despite the encouraging signs of developing writing competence, the failure of the DRP to 
show evidence of reading improvement, data which is supported by the below-average writing 
synthesis subscores, is of great concern as we look at our students' potential for academic success. 
Good reading skills become increasingly important in mastering the content of sophomore level 
courses; perhaps further required critical reading courses should be required along with the second level 
writing courses needed to transfer to upper division colleges. 

Student Surve y Materi als and Procedures. We have surveyed students in both integrated 
and control classes in all six semesters; however, only data from the last three semesters will be 
reported here. There were three major difficulties: getting students to take the survey seriously, 
getting students to return the surveys, and matching pre and post surveys for individual students while . 
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Table 2. Essay Quality Categories by Group 



Essay 1 


0-9.9 Weak 


10-19.9 Low 


20-29.9 Good 


30-40 

Excellent 


Number 


Exper. n 


19 


34 


20 


12 


85 


% 


22.4 


40.0 


23.5 


14.1 




Control n 


10 


25 


7 


3 


45 


% 


22.2 


55.6 


15.6 


6.7 




Essay 1 : Syn. 


0-1.9 Weak 


2-3.9 Low 


4-5.9 Good 


6-8 Excellent 




Exper. n 


20 


32 


15 


18 


85 


% 


23.5 


37.6 


17.6 


21.2 




Control n 


11 


19 


10 


5 


45 


% 


24.4 


42.2 


22.2 


11.1 




Essay 1 :Anal. 


0-1.9 Weak 


2-3.9 Low 


4-5.9 Good 


6-8 Excellent 




Exper. n 


20 


26 


26 


13 


85 


% 


23.5 


30.6 


30.6 


15.3 




Control n 


8 


21 


11 


5 


45 


% 


~ ' 17.8 


46.7 


24.4 


11.1 




Essay 2 


0-9.9 Weak 


10-19.9 Low 


20-29.9 Good 


30-40 Excell. 




Exper. n 


9 


36 


24 


16 


85 


% 


10.6 


42.4 


28.2 


18.8 




Control n 


10 


21 


9 


5 


45 


% 


22.2 


46.7 


20.0 


11.1 




Essay 2: Syn. 


0-1 .9 Weak 


2-3.9 Low 


4-5.9 Good 


6-8 Excellent 




Exper. n 


12 


30 


21 


22 


85 


% 


14.1 


35.3 


24.7 


25.9* 




Control n 


10 


16 


14 


5 


45 


% 


22.2 


35.6 


31.1 


11.1 




Essay 2: Anal. 


0-1 .9 Weak 


2-3.9 Low 


4-5.9 Good 


6-8 Excellent 




Exper. n 


7 


32 


23 


23 


85 


% 


8.2* 


37.6 


27.1 


27.1 




Control n 


10 


17 


12 


6 


45 


% 


22.2 


37.8 


26.7 


13.3 
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Table 3. Student Communication In / Outside of Class from Student Survey 



— 


History 5 
Integrated 
Class 


History 5 
Control 
Class 


Political 
Science 1 
Integrated 
Class 


Political 
Science 1 
Control 
Class 


Nutrition 10 
Integrated 
Class 


Nutrition 10 
Control 
Class 


% of Enrolled 
Students 
Responding at 
Semester End 


46% 


61% 


55% 


42% 


65% 


44% 


Average. 

Times/Week 

Respondents: 














Talk to Teocher 
In Class 


6.15 


.78 


2,59 


2.58 


1.68 


.95 


Talk to Teacher 
Outside Class 


.62 


.42 


1.26 


.53 


.36 


.59 


Teacher Talked 
to Student in 
Class 


3.69 


.53 


2.94 


2.05 


1.55 


.51 


Talked with 
Students in 
Class 


5.77 


2.3 


3.38 


3.13 


2.27 


2.27 


Talked with 
Students Out 
of Class 


3.23 


.85 


2.12 


1.89 


1.27 


1.27 


Asked/Contrib. 
In Class 


4.77 


.85 


2.79 


2.84 


1.64 


.78 


Group Work In 
Class 


5.23 


.17 


2.62 


1.58 


1.59 


1.04 


Group Work 
Out of Class 


1.46 


.30 


.56 


.16 


.09 


.32 
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retaining anonymity. The latter problem was unsolvable; FIPSE Project Director Nan Wishner opted for 
anonymity in hopes of getting some honest student feedback. The survey she designed was used in 
the last three semesters and is attached in Appendix C. 

Faculty had their choice as to whether they wanted to administer the surveys in class or out. 
Surveys in History 5 were done out of class, while Political Science and Nutrition were done in class. 
Even in class, a number of students will be absent on any given day, and if surveys are anonymous, it 
is hard to get absent students to compelte the survey later. Thus, we have considerably fewer surveys 
than students enrolled in some classes (See Table 3). Further, we cannot tell which students' surveys 
in the control classes come from students taking English 1; indeed, the number of control students in 
the last three semesters has been so small that the preponderance of survey data in the control classes 
comes from students who have already~or not yet-taken English 1 . For all of these reasons, we can 
get a sense of student opinion from the surveys, but they do not offer statistically sound evidence. 

Student Survey Resu lt s . The survey question analyzed for Table 3 deals with a major 
concern of this FIPSE Project: active student participation in class and out. We hoped this classroom 
approach would foster student/teacher and student/student interactions. Our goal was to create more 
of a sense of an academic community than is usually found in lecture classes. Evidence that that 
occurred appears in the History 5 student survey results. The control class was a strictly lecture style 
class, and we much less talk in this class compared to its integrated class counterpart. It must be 
noted, however, that the percentages of talk in the integrated History 5 class were influenced by three 
students of the 13 responding, each of whom reported 10 to 20 instances of talk in each week. Thus, 
we can see how different student interpretations of the question resulted in vastly different responses. 
Since this class met three times a week, these students were counting their multiple participation 
throughout the class hour. The frequency with which other students responded that they talked once, 
twice, or three times a week suggests that they recalled participating in every class period, but without 
counting each instance. Still, the contrast in participation in greatest in this pair of classes. 

Where the control classes were typical lecture classes, but with some group work included 
frequently, as in the Nutrition 10 control class, the students' sense of participation is more nearly equal 
to that found in the integrated class. Whether the fact that the control met three times a week and the 
integrated class met twice a week influenced student recollection of pariticipation is unknown. The 
instructor of the Political Science control class pointedly makes his class a discussion-based rather than 
lecture-based class. Student comments on what is valued frequently noted they like that quality about 
this political science class. It did make this control class more similar in nature to the integrated class 
than in either of the other two classes in this survey. 

In addition, there are the cases when students don't do what the instructors expect them to. 

In my integrated Nutrition 10/English 1 class, there were two debate assignments, where students 
were expected to plan their debate strategy and information with other students on their team. Yet 
only one student in our class reported working with the group outside of classl The students did not 
make the outside of class connections that we expected them to. 

Further student comments from the surveys will be included in the Final Discussion. 



Fill and Retention Analysis 

When this Project was in the planning stages, two questions we had were whether students 
would be interested in enrolling in the integrated classes and whether this integrated format would 
influence student retention. The following data will show that students enroll in integrated classes at 
a somewhat lower rate, but are retained at approximately the same rate as in the total of all sections of 
English 1 and of the content area courses. 

For this analysis, all of the non-integrated sections of following content courses were included 
during the semester in which the integrated section was offered: History 17, Human Development 38, 
Biology 15, Social Science 22, History 5, Political Science 1, Nutrition 10, CIS 1, and Art History. 
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Courses 


Number of 
Sections 


Percent of Fill 


Percent Retention 


Total Separate English 1 


200 


94.4 


71.7 


Total Integrated English 1 


12 


81.4* 


70.3 


Total Separate Content Areas 


87 


100.7 


72.4 


Total Integrated Content Areas 


12 


81.3* 


69.3 



“Fill” means the number of students who are enrolled at the first census week about the fifth week of 
the semester as a percentage of the contractual class maximums. There is a statistically significant 
difference in fill rate { t-test ). “Retention” means the number of students who earned a grade of A, B, 
C, D, F, CR, or NC; it does not include the number of students who dropped. The retention rate for the 
integrated classes is not significantly different from the separate class retention rates. Given that there 
has been considerably more reading assigned in the Project integrated classes, and that some students 
have viewed these classes as harder and more work than separate classes, maintaining an equal 
retention rate is quite good. One additional factor is that students may be more reluctant to drop six 
units worth of courses than the three units of one separate course. 

Follow-up Study 

After chance meetings on campus with early integrated class students yielded great reports of 
their further academic success, we wanted to discover whether participation in an integrated class led 
to a higher level of re-enrollment in college courses and a greater degree of success in subsequent 
English and related content area classes. Diane White did mail and telephone surveying of former 
students in the second year of the FIPSE Project, unfortunately, without getting much feedback. A 
second try in this third year yielded somewhat more information. 

Rerenio Ilmen t_in. College Courses. 

Diane White and Susan Messina tracked students who had completed an integrated class 
through subsequent semesters and found the re-enrollment patterns listed in Table 4. Although 
students did continue to re-enroll at Solano in varying numbers of later semesters, data was too sparse 
and irregular to record for semesters beyond the one immediately following the semester in which the 
integrated class was taken. Thus, the enrollment patterns in Table 4 reflect enrollment in the semester 
immediately following the one in which the integrated or control class was taken. 

We wanted to know whether students became more interested in taking further courses in the 
content area and whether they took more English courses as a result of their experience in an 
integrated class. Examination of the specific courses students took in these two areas revealed that 
College General Education Requirements probably have a greater influence on course choice than 
personal interest does. In general, the content area courses chosen by students reflected the breadth 
demanded by the General Education Requirements and the advanced level English courses required by 
near-by four-year colleges to which our students frequently transfer rather than a more advanced 
history or biology course. 

The data in Table 4 show that there is little difference between the re-enrollment patterns of 
integrated class students and the control class students; indeed, in total, a somewhat higher 
percentage of control class students re-enrolled at Solano College, in another content area course, and 
in more English courses. 

Fnllnw-up Snrve y Res ults. 

In Spring Semester 1995, the last semester of this FIPSE Project, Diane White mailed 135 
follow-up surveys to students who had completed an integrated class or a Project control class in the 
previous five semesters. Thirty surveys were returned, 21 from integrated class students, and 9 from 
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Table 4. Percentage of Project Students Who Re-Enrolled in Semester Following 
FIPSE Evaluation 





Fall 1992 


Spr. '93 % 
Enrolled SCC 


Spr. '93 % 
Enrolled in 
OtherHistory 


Spr '93 % 
Enrolled in 
Other English 


History 17 Exp. 


22 


96% 


4% 


39% 


Control 


23 


77% 


23% 


50% 




Spring 1 993 


Fall '93 % 
Enrolled SCC 


Fall '93 % 
Enrolled in 
Other Biology 


Fall '93 % 
Enrolled in 
Other English 


Biology^ 5~txp. 


1 0 


40% 


10% 


0% 


Control 


8 


75% 


0% 


50% 




Fall 1993 


Spr. '94 % 
Enrolled SCC 


Spr. '94 % 
Enrolled in 
Other Sociology 


. Spr. '94 % 
Enrolled in 
Other English 


Social Science 
22 Exp. 


10 


80% 


10% 


50% 


Control 


6 


67% 


17% 


0% 




Spring 1 994 


Fall '94 % 
Enrolled SCC 


Fall '94 % 
Enrolled in 
Other History 


Fall '94 % 
Enrolled in 
Other English 


History 5 Exp. 


23 


82% 


18% 


59% 


Control 


1 


0% 


0% 


0% 




Fall 1994 


Spr. '95 % 
Enrolled SCC 


Spr. '95 % 
Enrolled in 
Other Poli. Sci. 


Spr. '95 % 
Enrolled in 
Other English 


Political 
Science 1 Exp. 


27 


70% 


7% 


44% 


Control 


5 


80% 


0% 


60% 


Average of 5 
Semesters Exp. 


92 


68% 


8% 


33% 


Average of 5 
Semesters Con. 


43 


74% 


22% 


40% 




M O 

ii. 
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control class students, a 22% return rate. See Appendix C for sample surveys. 

All but one of the 30 students are currently enrolled in college. Three-quarters of the control 
group students have transferred to other colleges, while 85 % of integrated class students are still 
enrolled at Solano College. This finding somewhat contradicts the first semester after enrollment 
patterns described above. Students from both groups responded that their English class has helped 
them in other English and content area classes and that the content area classes have provided useful 
background in other classes. 

Both integrated and control class responders have about two-thirds “discussers” and one-third 
who don't like to contribute to class discussion. Thus, willingness to discuss issues in class does not 
seem to be related to class instructional method. All but one of the thirty responders felt either 
“somewhat” or “very" comfortable expressing ideas and asking questions in class. One former 
integrated class student said, “At first it was uncomfortable to express my ideas, but it became second 
nature.” It is possible, though, that only the people who are comfortable expressing ideas were 
motivated to respond to this survey. 

An average of 85% of both groups said they would like to read more about the content area 
subject, and over 90 % said yes to reading more about English. Approximately two-thirds of both 
groups would like to take more courses in the content areas related to the FIPSE Project and in English. 

Thus, what we found is a great deal of similarity in the responses from former integrated class 
and control class students. It may well be that there is more motivation to respond to a survey in 
people who are enthusiastic about the courses they have taken and the learning they feel they have 
gained than in those who are less positive. Still, it is encouraging to hear so many positive responses 
about their Solano College experiences from these students. And the fact that twice as many former 
integrated class students as control class students made the effort to respond says a great deal about 
the long lasting positive effect of the Project classes. 

Eollo.wdup Writing^Samples. As part of our evaluation of the effectiveness of the integrated courses 
we collected writing samples from the instructors of students who had participated in the evaluation in 
previous semesters. Specifically, we asked both English and content area instructors to photocopy at 
least one piece of substantive writing produced by a student who had been in either the experimental 
or control group. In addition, we told instructors that we preferred to have a piece of writing that 
contained their comments and that had been assigned a grade. Students who agreed to allow their 
work to be reviewed by our evaluation team signed release forms authorizing the photocopying and 
assessing of their papers. 

Our first efforts at obtaining writing samples were not successful for several reasons. One 
reason for the lack of samples was that we originally made our requests near the end of the semester. 
We learned that many instructors, particularly in the content areas, assigned most of the writing for 
their courses by mid-term and had already given papers back to students. This is understandable given 
that the average class size in many content areas is fifty students. Assigning papers at the end of the 
semester would make it difficult for instructors to complete their final 

grades on time. Another reason why we had difficulty obtaining students’ writing samples from content 
area instructors is that these teachers often did not actually have students write substantive essays. 
Very often the type of writing required in content area courses is very short 

pieces (e.g., the short answer "essay” or identification of terms). In addition, many samples that 
content area instructors submitted contained few, if any, comments. For English instructors 
who often do assign essays at the end of the semester, this request put an additional end-of-the 
semester burden on them. Thus, it was not surprising that we had few instructors volunteer to submit 
student samples. 

In the final semester of the project the requests were sent out in the thirteenth and fourteenth 
weeks of the semester. Not surprisingly, our rate of return was higher. We received 
fifteen essays in this last semester. However, upon assessing these writing samples and examining 
the distribution of grades, we found that no conclusions about student performance could be reached. 
There were several major issues that made a comparative analysis impossible, including the following: 
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a wide variation in topics; the diversity of instructors' expectations and evaluation criteria; a range of 
conditions and assignment parameters under which the work was produced (e.g., in-class exam or term 
paper). 

First, there were wide variations in the topics that different instructors posed that made it 
impossible to compare students* writings. These variations were easily detected by examining two 
essays; one from a History of Western Civilization course, and one from an English Literature course. 
The instructor of Western Civilization asked students to explain how and why patriarchal societies 
developed in western culture. By contrast, one English instructor asked for an essay in which the 
student was to describe several themes in a short story. The nature of these assignments is 
considerably different, as the history essay required a specific type of analysis, while the English essay 
was more descriptive. This basic difference made it hard to compare the two students* work, because 
they were being asked to perform two different intellectual tasks. 

The comments that instructors wrote at the end of these papers reflect both the differences in 
the nature of the topics and the instructors' varying expectations. At the end of the student essay on 
the development of patriarchy in the west, the instructor wrote, “You use the varying sources of 
evidence. ..but need to include more discussion of the actual causes of these changes, such as complex 
agriculture, population growth, invasions.” Clearly, this instructor assigned a cause-effect essay and 
expected to see a cause-effect analysis. On the other hand, at the end of the essay describing a short 
story, the instructor wrote, “You include important points in your paper.... Don't be afraid to fully 
examine them. Learn to trust your Instinct and intelligence.* 1 The instructor also observed that literary 
analysis should be presented in the “present tense.” These comments suggest that perhaps the 
instructor was more concerned about allowing the student to shape the assignment in a way that 
developed the student’s confidence, rather than in creating a specific analytical task. Again, these are 
fundamental differences in topic that relate to instructor approach but which make comparisons of 
students* work difficult. 

Another type of "essay" assignment that we encountered via students’ writing samples reflects 
huge variations in instructors* definitions of an essay. For example, in one final exam assignment sheet 
that was submitted along with a writing sample for a Philosophy class, students were asked to write 
on ten out of a possible twenty topics. The instructor used words like "state" and “identify” as 
directives for these topics. Clearly, these kinds of instructions allow students to relay bits and pieces of 
information, but do not call upon the student to analyze the material. Thus, it would be pointless to 
attempt to compare an essay that "states the five fundamental concepts or ethical principles of 
Confucianism with the essay that required students to explain how and why patriarchy developed in 
western cultures. The nature of the two topics can and did produce two extremely different types of 
writing that defied comparison.. 

Some of the writing samples submitted did not allow us to determine the topic posed or the 
assignment given. For example, one sample that was collected was a "research paper” on the “middle 
passage" (i.e., the transporting of slaves from Africa to the American colonies). However, the 
information presented was extremely general in nature and no sources were documented. Thus, 
determining the exact task the instructor had posed to the student was impossible. 

This research paper sample also reflects huge variations in instructor expectations and 
evaluation criteria. This paper contained no thesis statement, did not document any sources used, 
offered no analysis of the topic, and contained numerous errors in basic grammar, academic format, 
and punctuation. The assigned grade for this paper was 100 points out of 100 possible, an "A." The 
instructor commented that it was "well written” and "what writing ought to be.” On the other hand, 
another student who submitted a research paper on the causes of the American Civil War began with a 
clear thesis, was well organized, contained few errors in grammar or format, documented most of the 
sources used, and attempted to follow through on the line of reasoning offered in the thesis. The grade 
assigned to this essay was 92 points out of 100 possible. The instructor commented that the primary 
sources used were not documented correctly and that the impact of some of the causes mentioned 
were not fully explained. Obviously, by examining these two research papers we can see significant 
differences in instructor expectations and evaluation criteria. 
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Another significant problem in comparing students' essays was the difference in conditions and 
parameters under which they were to write. For example, one student sample was a final exam for a 
Journalism class.. Students were to write on three essay topics in a two-hour final exam session. 
Understandably, the conditions under which this student wrote would not allow him to produce an 
essay that would be as fully developed as the student who had a week to write an essay explaining the 
structure of a specific poem. Again, comparison would be impossible. 

Student comments from the surveys were quite revealing. Much of what students from the 
linked classes reported reflected a positive learning experience. For example, students were asked to 
describe ways in which the skills and/or information that they learned in the linked classes helped them 
in other areas. One student reported that "essay writing and research was easier." Another student 
reported that "they gave me better writing skills and helped me with my own desires and motivations." 
Students also responded positively when asked about discussing ideas with other students. Many from 
the linked courses commented upon the high degree of student interaction. For example, one student 
reported that, there was "great camaraderie among classmates.” Other students who were surveyed 
made very similar remarks noting that they "felt more comfortable" speaking and that "others who 
normally would not have spoken up in class did speak up in this class.” A few reported that they had 
become more involved with group learning after taking the linked courses. However, it was very 
apparent that many did not have this sense of community in other academic settings. One student who 
now attends UC Davis remarked that Solano was a "more comfortable environment.” 

These comments clearly show that we did succeed at helping students create a productive and 
positive learning environment in which they develop both skills and confidence. Therefore, even if the 
academic environment in general is not conducive to creating "community" and “confidence," students 
leave the linked courses with an alternative paradigm for learning, and as Parker Palmer has said, "a 
way of being in the world." So, if ever again presented with a community oriented environment, it is 
likely that students from the linked classes would thrive. 

Conclusions 

We began this Project with great optimism and energy, and during the course of its six 
semesters we have experienced some wonderful moments of learning with our students. We are 
happy that our essay writing evaluation produced some evidence for the success of this pedagogical 
method, though disappointed that the reading evaluation did not reveal substantive changes in our 
students' abilities, especially since a close reading of texts has been such an important feature of all of 
the integrated classes. Still, we each saw the individual students whose scores showed great 
increases and shared their joy when they realized what they had accomplished. 

As experienced teachers we recognize that students do not always reveal what they have 
learned during the single semester that we see them; often the skills emerge in subsequent semesters, 
though we could not necessarily see it in our small sample of papers written by former integrated class 
students. All of us who have taught these integrated classes have experienced hearing from “alums” 
how much they learned, how much more thoughtful they have become about their learning. 

However, the one factor that we have not explored in this evaluation is an explanation of why 
this approach to teaching and learning did not work for some students. Two factors I can hypothesize 
from my own two classes in this Project are students' inadequate reading skills and inadequate time to 
devote to study. Perhaps further study of the students' learning process in this integrated learning 
environment can identify some evidence to explain the lack of success some students felt. 

On the whole, students ended the classes with an enthusiastic call for more integrated classes: 

“Why aren't there more coordinated studies programs?" 

“I have had a lot of fun and gotten an education at the same time. If any other coordinated 
study classes are made available, I would take them.” 

“Great class. Hope to see and take more in the future.” 
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Perhaps we will see more of the positive learning experiences started in these integrated classes as we 
develop our coordinated studies (our new name for theme-based, integrated classes) program and 
expand it to developmental and sophomore level classes. The evidence of pedagogical success found 
through this evaluation strengthens our surety that integrated studies offers opportunities for real 
learning. As students have a chance to take more than one such class, they will be able to develop 
their literacy and learning in more visible ways. 

One final note that must be added, though we did not have a formal mechanism for evaluating 
it, is the tremendously positive collegiality that developed among the Project faculty. Unexpected 
friendship, respect, and rejuvenation came out of the various partnerships: teaching pairs, essay 
scoring teams, part-timers and full-timers. And I am deeply indebted to Rob Simas and his secretary, 
Carol Duley, for the imagination and help they contributed to this evaluation. Most importantly, we 
appreciate the support of the Fund for Post-Secondary Education; without it we could not have 
embarked on this learning journey. 




? 



APPENDIX A: 



PRE AND POST ESSAY TOPICS 
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English 1/History 17 



Essay 1 September 1992 



DIRECTIONS: 

In the videotape "Roanoke," the two Native Americans Manteo and 
Wanchese eventually take different views of the English 
settlement on Roanoke Island, with Manteo becoming a stronger 
friend to the English. We hear them and White, an Englishman, 
expressing the following views: 

Manteo: "We can flourish together — whites and natives 
alike . " 

White: "Have we not done some wonders here — Englishmen 

[and] native of this land, living side by side." 

Wanchese: "We cannot live together on this island." 

Considering the nature of the English and Native American 
cultures and the experiences the two groups had on Roanoak, did 
the relationship between the English and the Native Americans 
necessarily have to turn out the way Wanchese described it, or 
was Manteo and White's dream possible? In your essay, take a 
position and support it with evidence from the article "Roanoke 
Lost," the video "Roanoke," and other class readings, materials, 
and discussion. 





English 1/History 17 



Essay 2 December 1992 



DIRECTIONS 

Write an essay on the following topic on the attached lined 
pages. If you need more room, ask the instructor for more lined 
paper and staple it to the back of your essay; be sure to number 
any added pages. You may use your one page of notes; also stple 
it to the back of your essay when finished. Please write in ink 
if possible; neat cross-outs and corrections are acceptable. 



TOPIC 

Both the video "Ethnic Notions" and the article "The Lives of 
Slave Women" examine images and experiences of female slaves. 

Take a position on the following question and use evidence from 
these sources as well as from class discussion and other class 
readings in developing your essay. 

Was the "Mammy" image an accurate or inaccurate reflection of the 
real lives of slave women in the antebellum South? What does 
this image reveal about the relationships between blacks and 
whites in antebellum America ? Support your position with reasons 
and examples. 
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English 1/Biology 15 Essay l February 1993 

DIRECTIONS: 

Using information from class notes, your biology text, "A 
Crossbill's Twist of Fate" by Benkman, "The Science That Reasons 
Why" and "Every Species Has its Niche" by Colinvaux, and other 
class readings, write an essay on the topic below. 

Write your essay on the attached lined paper. If you need more 
room, ask the instructor for more lined paper and staple it to 
the back of your essay; be sure to number any added pages, but 
don't write your name on them. You may use your one page of 
notes; also staple it to the back of your essay when finished. 
Please write in ink if possible; neat cross-outs and corrections 
are acceptable. 

TOPIC: 

"The Genetics of the Garter's Getaway" describes different 
coloration patterns found in a single species of garter snake. 

Explain the selective processes and ecological principles 
that are at work in the polymorphism of the northwestern 
garter snake; in other words, in what way has natural 
selection produced color patterns that seem to suit the 
western garter snake to its niche? In addition, how is the 
adaptation of the northwestern garter snake similar to or 
different from other examples you have studied? 



English 1/Biology 15 



Essay 2 May 1993 



DIRECTIONS: 

Using information from "Russian Wheat Aphid" by Gonzalez, 

Summers, and Qualset, "Why All the Fuss about Prserving Wild 
Stocks of Salmon and Steelhead?" by Higgins, "The Biological 
Control of Dung" by Waterhouse, your biology text, and your class 
notes, write an essay on the topic below. 

Write your essay on the attached line paper. If you need more 
room, ask the instructor for more line paper and staple it to the 
back of your essay; be sure to number any added pages, but don't 
write your name on them. You may use your one page of notes; 
also staple it to the back of your essay when finished. Please 
write in ink if possible; neat cross-outs and corrections are 
acceptable . 



TOPIC: 

Identify and explain the biological problems involved in the 
Russian Wheat Aphid infestation of single varieties of grains. 
Evaluate the possible solutions to this problem according to 
their biological consequences. Compare this problem and its 
solutions to other examples you have studied. 



English 1/Social Science 22 
DIRECTIONS: 



Essay 1 September 1993 



Write an essay on the topic below using information from class 
discussion, the video about Sherrie Pietranico (the Rutgers 
University graduate student) , the reading selections from 
Schaefer's Racial and Ethnic Groups . Betances' "Race and the 
Search for Identity," and Piper's "Passing for White, Passing for 
Black," and— -if you wish — your own experience or observations. 

Write your essay on the attached lined pages. If you need more 
room, ask the instructor for more lined paper and staple it to 
the back of your essay; be sure to number any added pages, but 
don't write your name on them. You may use your one page of 
notes; also staple it to the back of your essay when finished. 
Please write in ink if possible; neat cross-outs and corrections 
are acceptable. 



TOPIC: 

In the last century in the United States, the concept of "race" 
has emerged and been used to categorize people. This 
categorization has produced both negative and positive 
consequences for social groups and for individuals. 

Considering the concepts of "race" discussed by Schaefer and 
Betances, contrast a) how Piper and Sherrie Pietranico perceive 
themselves racially and b) how others perceive them racially. 
Additionally, you may include one example from your own life or 
observation to support your points. 
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English 1/Social Science 22 
DIRECTIONS: 



Essay 2 December 1993 



Write an essay on the topic below using information from class 
discussion, the short video selection from Where the Spirit 
Lives, the reading selections (Saidell, H Grow.ing Up in Linguistic 
Limbo," Fordham and Ogbu, "Black Students' School Success: Coping 
with the 'Burden of Acting White,'" and Tan, "Mother Tongue") 
and — if you wish — your own experience or observations. 

Write your essay on the attached lined pages. If you need more 
room, ask the instructor for more lined paper and staple it to 
the back of your essay; be sure to number any added pages, but 
don't write your name on them. You may use your one page of 
notes; also staple it to the back of your essay when finished. 
Please write in ink if possible; neat cross— outs and corrections 
are acceptable. 



TOPIC: 

Our class discussion, the video, and the three articles have all 
focused on education, language, and the family as we explored 
various dominant group strategies to control minority group 
assimilation into mainstream American society. 

(a) Contrast the strategies used by minorities as they either 
resist or accept assimilation. (b) Also describe specific 
consequences for minorities, their families, and their friends. 
Be sure to support your points with examples from the articles 
and video. Additionally, you may include one example from your 
own life or observation to support your points. 
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English 1/History 5 
DIRECTIONS: 



Essay l February 1994 



Write an essay on the topic below using information from your 
textbooks, class notes, the Voltaire essay, and other class 
readings and discussions. 

Write your essay on the attached lined pages. If you need more 
room, ask the instructor for more lined paper and staple it to 
the back of your essay; be sure to number any added pages, but 
don't write your name on them. You may use your one page of 
notes; also staple it to the back of your essay when finished. 
Please write in ink if possible; neat cross-outs and corrections 
are acceptable. 



TOPIC: 

The thinkers of the Scientific Revolution challenged and replaced 
the medieval view of the universe and laid the groundwork for the 
fundamental innovations in political and religious theories of 
the Enlightenment. 

Write an essay in which you 1) explain how the thinkers of the 
Scientific Revolution replaced the medieval view of the universe 
and then 2) explain how Enlightenment thinkers shaped political 
and religious thought according to the concepts produced by the 
Scientific Revolution. 

Organize your essay and make an effort to be as specific as 
possible, offering details and examples to support your answers. 
Be sure to mention specific thinkers and their contributions. 
Include Voltaire in your answer. 
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English 1/History 5 



Essay 2 May 1994 



DIRECTIONS: 

Write an essay on the topic below using information from your 
textbooks; class notes; the three handouts: Churchill, "Blood, 
Toil, Tears, and Sweat,” Hoess, "Commandant of Auschwitz,” 

Wiesel, "Reflections of a Survivor"; and other class readings and 
discussions . 

Write your essay on the attached lined pages. If you need more 
room, ask the instructor for more lined paper and staple it to 
the back of your essay; be sure to number any added pages, but 
don't write your name on them. You may use your one page of 
notes; also staple it to the back of your essay when finished. 
Please write in ink if possible; neat cross-outs and corrections 
are acceptable. 



TOPIC: 

The policy of appeasement pursued in the 1930 's by the British 
and French failed to prevent World War II in Europe. This 
failure eventually led to the "Final Solution." 

Write an essay in which you identify and explain the key events 
of appeasement resulting from this British and French foreign 
policy and explain how these events and policies led to the 
"Final Solution.” 

In your essay be sure to define the terms "appeasement" and 
"Final Solution." Include and explain how examples from the 
readings support your points. 
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DIRECTIONS; 



Write an essay on the topic below using information from your 
textbook; class notes and discussion; and the handouts: Locke, 
"Of the Beginning of Political Societies" from The Second 
Treatise of Civil Government . Jefferson, "The Declaration of 
Independence," Stanton, "The Seneca Falls Declaration," Truth, 
"Ain't I a Woman?" and Henry, "In Defense of Elitism." 

Write your essay on the attached lined pages. If you need more 
room, ask the instructor for more lined paper and staple it to 
the back of your essay; be sure to number any added pages, but 
don't write your name on them. You may use your one page of 
notes; also staple it to the back of your essay when finished. 
Please write in ink if possible; neat cross-outs and corrections 
are acceptable. 



TOPIC; 

Several types of "equality" have been discussed in class and in 
the readings. We have considered political equality, human 
equality, equality of opportunity, equality of power, and 
intellectual equality. 

Compare these different conceptions of equality using the 
assigned readings and your own views to illustrate your points. 
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DIRECTIONS: 



Write an essay on the topic below using information from your 
textbook (Chapters 4 and 5) ; class notes and discussion; the 
video on the Dolfman case; and the handouts: Nat Henthoff, "Free 
Speech on the Campus," B. Ehrenreich, "Teach Diversity with a 
Smile," and W. Henry, III, "The Politics of Separation." 

Write your essay on the attached lined pages. If you need more 
room, ask the instructor for more lined paper and staple it to 
the back of your essay; be sure to number any added pages, but 
don't write your name on them. You may use your one page of 
notes; also staple it to the back of your essay when finished. 
Please write in ink if possible; neat cross-outs and corrections 
are acceptable. 



TOPIC: 

The history of the United States has often reflected the tension 
between civil liberties and civil rights. The writers of the 
assigned readings address this tension as they consider issues of 
discrimination, diversity, political correctness, and equality. 

In "The Politics of Separation," William A. Henry, III says, "In 
place of freedom of speech has come a demand for freedom from 
speech, if that speech is deemed offensive by any victim group." 

How does this quotation exemplify the tension between the civil 
liberties guaranteed by the First Amendment (free speech and 
press) and the civil rights guaranteed by the Fourteenth 
Amendment (equal protection of the law)? Support your position 
with specific examples from all of the assigned readings, the 
Dolfman video, and class lectures and discussions. 



DIRECTIONS 



Write an essay on the topic below using information from your 
textbook (Chapter 1); class notes and discussion; the videos on 
the food advertisements and Eat. Drink, and Be Wary ; and the 
handouts: "Nutrition and Modern Lifestyles," "A Nation Playing 
with its Food," "Fat Times," "Are You Feeding Your Baby Too 
Much?" "The Changing American Diet," and "When Food is Love." 

Write your essay on the attached lined pages. If you need more 
room, ask the instructor for more lined paper and staple it to 
the back of your essay; be sure to number any added pages, but 
don't write your name on them. You may use your one page of 
notes; also staple it to the back of your essay when finished. 
Please write in ink if possible; neat cross-outs and corrections 
are acceptable. 



TOPIC: 

The food choices we make are influenced by cultural, 
psychological, economic, and health factors. We microwave ready 
to-eat food, get and give our loving from the Sara Lee oven, 
spend $ .99 for a McDonald's special, but hope to be healthier 
than our parents' generation. Changes in American family 
structure, individual self-image, and lifestyles have led us to 
demand more convenient food choices. 

In your essay, explain how and why our demand for convenience 
foods is related to these cultural, psychological, economic, and 
health issues. Use specific examples from the readings, the 
videos, class discussion, and your own experience. 
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DIRECTIONS 



Write an essay on the topic below using information from your 
textbook; class notes and discussion; and the handouts: "Hunger 

and Global Issues," "Hunger and Undernutrition in America," 
"Hunger in California," "Lifeboat Ethics," "Nutrition and World 
Hunger," "World Hunger: Chronology and The Issues." 

Write your essay on the attached lined pages. If you need more 
room, ask the instructor for more lined paper and staple it to 
the back of your essay; be sure to number any added pages, but 
don't write your name on them. You may use your one page of 
notes; also staple it to the back of your essay when finished. 
Please write in ink if possible; neat cross-outs and corrections 
are acceptable. 



TOPIC: 

The articles and our class discussion point out that the adeguacy 
of the food supply has major nutritional consequences ranging 
from good health to hunger, disease, malnutrition, even 
starvation. Recall that we discussed the influences of politics, 
government, culture, technology, population, and economics on 
food demand and supply. 

In your essay, explain how poverty is related to food demand and 
supply in California in contrast to at least one of the following 
countries: Russia, Bosnia, Rwanda, or Somalia. Use specific 
examples from the readings, class discussion, and your own 
experience. ■ 
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APPENDIX B: 



ESSAY SCORING RUBRICS 



Q ? 
O IL 




SCORING RUBRICS: 



History 17: Survey of U. s. History to 1877 
Biology 15: Introduction to Biology 
Social Science 22: Minority Group Relations 
History 5: Western Civilization 

The scoring rubric is developed by the faculty team of integrated 
and control class instructors, the Evaluation Director, and two 
faculty — one English instructor, one content area instructor — who 
often teach the course being evaluated but are not teaching one 
of the project courses in the semester of the evaluation. 

The analytic categories and percentage of total score remain the 
same across all classes in the six-semester evaluation so that 
data can be compared across the curriculum. Note: The History 17 
rubric is slightly different because it combined Synthesis and 
Analysis. That proved to be too difficult to score accurately, 
so those two categories were separated in all subsequent 
semesters . 

Each group specifies the exact criteria for its evaluation; some 
teams have wanted more detail, others less. 



SCORING RUBRIC FOR HISTORY 17/ENGLISH 1 ESSAYS 



Fall 1992 



CONTENT 

This criterion includes 1) accuracy, 2) sufficiency, and 3) 
logical relevance and appropriateness of historical facts and 
concepts . 

0 3 6 9 12 

Lacking Low Average Good Excellent 

An excellent or good essay is accurate in thesis and support, 
offers a sufficient number of supports to convince the reader, 
and uses a variety of types of support which are appropriate for, 
and relevant to, the argument being made; lower rated essays are 
inaccurate to varying degrees, have insufficient or irrelevant 
support . 

1. Historical accuracy 

— Plausible, supportable position (even if it is not well- 
supported in this paper, give credit if it could have been 
accurately supported based on historical information) 

— Accurate supporting details (give credit on sliding scale 
based on proportion and importance of any inaccuracies) 

2. Sufficient number of examples and explanation given in 
development to convince reader of thesis position 

3. Relevance and appropriateness as support for thesis 

— Examples and explanation may include historical 
chronology, general characteristics of time period, 
cultures, and of the belief systems, significant themes and 
concepts, an understanding of issues and cause/effect 
relationships, precise terminology 

SYNTHESIS AND CRITICAL ANALYSIS 

This criterion includes 1) use of information and ideas from 
readings, video, and class discussion as well as writer 's own 
ideas 2) integrated in support of a logical, thoughtful, and 
evaluative explanation why the position taken in the thesis is or 
could have been true. 

0 3 6 9 12 

Lacking Low Average Good Excellent 

An excellent or good essay will make explicit exactly how and why 
the support proves the validity of the thesis through explanation 
and use of supports from the various sources; a lower rated essay 
may simply state historical facts or concepts in a supporting 
position, but not explain why they support, or it may use only 
lecture material. 




O 

<a) 



1. Inclusion of materials from all of the unit sources: video 
"Roanoke," reading "Roanoke Lost," themes and concepts from class 
discussion, writer's background knowledge, and other assigned 
class materials and texts. [Changed to include slavery readings 
for Essay 2 . ] 

— A good paper may take pieces of information bearing on a 
single aspect of the topic from several of these sources and 
weaves them together in a logical grouping. 

2. Explanation of why events occurred as they did, or why people 
acted as they did based on their beliefs and knowledge, either 
unique to a particular historical figure or common in time period 

— Lesser papers focus more on narrative of events, while 
superior papers explain why what happened did happen or 
could have happened in this particular time and place. 

— Good essays use explanations that are logical . 

— Excellent essays will demonstrate critical thinking that 
evaluates the participants, events, and themes rather than 
just reporting them and will contain original manipulation 
of ideas, not just parroting information and concepts from 
class sources. 

STRUCTURE AND ORGANIZATION (GLOBAL EXPRESSION) 

This criterion includes the 1) clarity, 2) specificity, and 3) 
logical coherence of the structure of the essay (i.e. thesis, 
support, and conclusion) . 

0 3 6 9 12 

Lacking Low Average Good Excellent 

An excellent or good essay will take a specific position based on 
the prompt, develop that argument logically, specifically, and 
creatively, and conclude with a point that follows logically from 
the thesis and support. Lower rated essays will lack clarity, 
specificity, or coherence, and may even lack one or more 
essential structural components (thesis, support, or conclusion) . 

1. Has organization and structure that is appropriate for 
argumentative essay to convey position clearly 

— Uses logical arrangement of support 

— Signals structure and coherence appropriately with reader 
in mind 

2. States a position which responds to prompt specifically, 
precisely, logically, and completely 

3. Shows logical consistency and coherence in developing line of 
thought so that position runs clearly throughout essay, logical 
and meaningful flow of ideas 

— Main idea statements may be present in topic sentence or 
conclusion of paragraphs which explain how material in each 
paragraph supports and develops thesis statement, relevance 
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of support to thesis 

— Conclusion which logically follows from thesis and 
development 



SENTENCE LEVEL EXPRESSION 

This criterion includes diction; sentence structure; and grammar, 
punctuation, and mechanics. 

0 12 3 4 

Lacking Low Average Good Excellent 

An excellent or good essay will include precise, appropriate word 
choices; variety, complexity, correctness, and appropriateness of 
sentence structure; and correct grammar, punctuation, and 
spelling. Lower rated papers will have incorrect or imprecise 
diction, errors in or simplistic sentence structure, or errors in 
grammar, spelling, punctuation, or mechanics which interfere with 
reader's easy comprehension of essay. 

1. Sentence structure 

— Logical coordination and subordination 

2 . Correct grammar 

— Appropriate consistent verb tenses, subject-verb 
agreement, clear pronoun reference and agreement, correct 
modifier forms 

3. Correct mechanics which do not interfere too much with 
reader's understanding of writer's intended meaning 

— Correct spelling, keeping in mind that essays are written 
without dictionaries and within a fifty-minute time limit. 

— Correct essay and paragraph format 
— Careful proofreading 

The notes following the criteria above are intended to be 
guidelines, not prescriptions. Individual papers will differ, 
and the reader must use his/her own judgment about the extent to 
which creative and other variant papers merit scores in each 
range. Not all characteristics listed must be present in every 
paper . 



SCORING RUBRIC FOR BIOLOGY 15/ENGLISH 1 ESSAYS 



Spring 1993 



CONTENT AND TERMINOLOGY 

Accurate knowledge of basic concepts 
Accurate terminology used fluently 
Sufficient, relevant, accurate details 

Sufficient, relevant, accurate examples, evidence 

Completeness of answer 

0 3 6 9 12 

Lacking Low Average Good Excellent Score 



SYNTHESIS 

Connecting different concepts accurately 
Using different sources, integration of sources 
0 2 4 6 8 

Lacking Low Average Good Excellent Score. 



ANALYSIS 

Application of concepts to new situation 
Drawing inferences 

Accurate cause-effect explanations 

0 2 4 6 8 

Lacking Low Average Good Excellent Score 



ORGANIZATION 

Clear focus, thesis, either explicit or implicit 
Line of thought clear and well-developed 
Logical sequencing, transitions 
Cohesiveness 

0 2 4 6 8 

Lacking Low Average Good Excellent Score 



MECHANICS 

Correct sentence structure clearly conveying meaning 
Clear, correct syntax 
Correct punctuation 

0 12 3 4 

Lacking Low Average Good Excellent Score 



READER'S INITIALS TOTAL SCORE. 
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SCORING RUBRIC for SOCIAL SCIENCE 22 /ENGLISH 1 ESSAYS 



Fall 1993 



Course/Unit Content (30 %) 

Understanding concepts 

Factual knowledge 
summaries 

references to materials of unit 
examples 

Completeness /sufficiency 

Frame=quality=frame + appropriate examples; prioritize 
relevant details. 

Completeness + sufficiency means either multiple relevant 
examples or variety of perspectives on an extended example. 

Accuracy 

Correct factual information, correct use of terminology 



Synthesis (20 %) 

Use of all (or multiple) sources and materials: readings, video, 
class lecture and discussion, and personal experience; range 
in scope of references 

Connecting the different sources to show the conceptual 

relationships among them (related to Frame in organization) . 
Relate concepts (link) — to outside-of-class reality. 
Integrating what they know — desired but not required. 



Analysis (20%) 

Analyze personal experience by relating to concepts. 

Analysis includes: 

1. Systematic definitions of specifics of the cases 
Defining features 

Relating features into structures 
Relating structures to context 

2. Explain how the theory accounts for data, which means that 
something has to come from the outside — the theoretical 
frame — how to sub-group working people, for example. 

3. Take a critical stance, make evaluations and judgments based 
on a structure of values, standards, assumptions which are 
explicitly explained/articulated within the frame. 

4. Create/reevaluate frame/structure — desirable but not 
required . 



Organization (20%) 

Frame: establish a theme and a structure in introduction and 

conclusion; create thesis and follow throughout essay; 
component parts of essay in balance (more attention to 
development than to introduction, for example) ; show control 
of whole plus parts. 

Thesis — either stated or implied 

Paragraphing based on units of thought 

Logical sequencing based on frame 

Transitions showing shifts and relationships 

Introduction and Conclusion 



Mechanics (10%) 

Spelling — most commonly used words spelled correctly — few errors 
in general vocabulary 

Essential punctuation identifying sentence boundaries 
Clear syntax (word order within sentences) 

Correct sentence structure 

In scoring, consider number and importance of errors and degree 
to which they interfere with smooth reading and comprehension of 
essay . 

For example, if there are 1-2 mechanical errors, score 4 
points in rating. 

Without explicitly counting errors, reduce score as number 
of mechanical errors increases. 
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SCORING RUBRIC FOR HISTORY 5/ENGLISH 1 ESSAYS 



Spring 1994 



CONTENT AND TERMINOLOGY 

Accurate knowledge of basic concepts 
Accurate terminology used fluently 
Sufficient, relevant, accurate facts 

Sufficient, relevant, accurate examples, evidence 

Completeness of answer 

0 3 6 9 12 

Lacking Low Average Good Excellent Score. 



SYNTHESIS 

Connecting different concepts and facts accurately 
Using different sources, integration of sources 
0 2 4 6 8 

Lacking Low Average Good Excellent Score. 



ANALYSIS 

Application of concepts to the question (s) 

Drawing inferences 

Accurate cause-effect explanations 

0 2 4 6 8 

Lacking Low Average Good Excellent Score 



ORGANIZATION 

Clear focus, thesis, either explicit or implicit 
Line of thought clear and well-developed 
Logical sequencing, transitions 
Cohesiveness 

0 2 4 6 8 

Lacking Low Average Good Excellent Score. 



MECHANICS 

Correct sentence structure clearly conveying meaning 
Clear, correct syntax 
Correct punctuation 

0 12 3 4 

Lacking Low Average Good Excellent Score 



READER'S INITIALS TOTAL SCORE. 
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SCORING RUBRIC FOR POLITICAL SCIENCE 1 ESSAY 

Fall 1994 

CONTENT AND TERMINOLOGY 

Clear, elaborated definition of terms 
Use of terms logically in discussion 
Accurate knowledge of basic concepts 
Accurate, relevant, sufficient examples 
Accurate facts, dates, events 
Answers all parts of topic 
0 3 6 9 12 

Lacking Low Average Good Excellent 

SYNTHESIS 

Inclusion of all materials from unit 
Concept similarities and differences 
Bringing ideas from unit together 
0 2 4 6 8 

Lacking Low Average Good Excellent 

ANALYSIS 

Use definitions to explain concepts/ex. 
Apply defs. and concepts to other examples 
0 2 4 6 8 

Lacking Low Average Good Excellent 

ORGANIZATION 

Clear presentation of main point(s) 

Line of thought expressed, leads to concl . 
Paragraphs with clear concept, 
advancing line of thought 
~ Adequate development using concepts, 

terms, examples; more attention to 
development than to introduction 
Intro, sets up idea to be developed 
Transitions show shifts and relationships 
Conclusion follows from line of thought 
0 2 4 6 8 

Lacking Low Average Good Excellent 

MECHANICS 

Correct spelling of common words 
Few errors in general vocabulary 
Essential punctuation identifying 
sentence boundaries 

Clear syntax (word order within sentence) 
Logical sentence structure 
0 12 3 4 

Lacking Low Average Good Excellent 

READER'S INITIALS TOTAL SCORE 




NUTRITION 10 / ENGLISH 1 FIPSE EVALUATION 
SPRING 1995 

Essay I Scoring Rubric 



CURRICULUM UNIT CONTENT (30%) 

Define and focus on primary theme: 
convenience 

Inclusion and explanation of cultural, 
psychological, economic, and 
health issues 

Specific explanations of the 

relationship between convenience 
and the four issues 

Use of accurate facts and examples 

Lacking Low Average Good Excellent 

0 3 6 9 12 

SYNTHESIS (20%) 

Integration of sources to support 
assertions 

Connecting the four issues to concept 
of convenience and to each other 



Lacki ng 


Low 


Average 


Good 


Excellent 


0 


2 


4 


6 


8 


ANALYSIS 


(20%) 









Accurate causal relationships 
Application of concepts to specifics 
from readings, ads, and/or 
experience 

Extension of concepts to aspects of 
culture not covered in class 

Lacking Low Average Good Excellent 

0 2 4 6 8 

ORGANIZATION (20%) 

Clear line of thought throughout essay 
Introduction establishing main point 
Well-developed paragraphs based on 
. related units of thought 

Specific support for assertions 
Logical sequencing indicated by clear 

transitions 

Conclusion relating to main point 



Lacking Low Average Good Excellent 

0 2 4 6 8 

MECHANICS (10%) 

Generally correct sentence structure 
which does not hinder meaning 

Generally accurate spelling — most 
commonly used words spelled 
correctly — few errors in general 
vocabulary 

Generally accurate punctuation 



Lacking 


Low 


Av^-age 


Go^d 


Excellent 


0 


l 


2 


3 


4 


Reader ' s 


Initials 




Total 


Score 
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Beginning-of-Semester Anonymous Questionnaire 



To make sure classes at Solano are the best possible, we would like information about this 
English 1 /Political Science 1 Coordinated Studies class. Please help us by filling out this 
questionnaire. Your instructors will return it to the college Institutional Researcher. They will 
not read this form but will receive a typed report of students' comments after the semester ends. 
Please do ndt put your name on this form. 

1 . Circle how many semesters you have attended college, including this semester: 

one two three four more (please indicate) 

2. How long had you been out of school prior to this semester? Check one: 

continuing student (was enrolled in have been out of school 3-4 years 

school during spring 1 994) have been out of school 5-1 0 years 

was in school within the past year have been out of school more than 1 0 years 

have been out of school 1 -2 years 

3. Why did you take this section and/or these instructors for this course? 



4. Please list three of the main reasons why you took this class and rank them, with #1 being 
most important: 

1. 

2 . 

3. 



5. What information did you use to select this class (please check all that apply) 

counselor advice catalog description 

other students' advice other; briefly describe 

6. If you spoke with a counselor, what did s/he tell you about this class? 




b/cs f94 



Thank you for yor help! 



Beginning-of-Semester Anonymous Questionnaire 



To make sure classes at Solano are the best possible, we would like information about this 
Political Science 1 class. Please help us by filling out this questionnaire. Your instructor will 
return it to the college Institutional Researcher. S/he will not read this form but will receive a 
typed report of students' comments after the semester ends. Please do not put vour name on 
this form. 



1 . Circle how many semesters you have attended college, including this semester: 

one two three four more (please indicate) 

2. How long had you been out of school prior to this semester? Check one: 

continuing student (was enrolled in have been out of school 3-4 years 

school during spring 1 994) have been out of school 5-10 years 

was in school within the past year have been out of school more than 1 0 years 

have been out of school 1 -2 years 

3. Why did you take this section and/or this instructor for this course? 



4. Please list three of the main reasons why you took this class and rank them, with #1 being 
most important: 

1 . 



2 . 



3. 



5. Are you taking any English classes this semester? yes no 

If yes, which one(s) 

English 4 (Critical Thinking) English 370 (English Fundamentals) 

— English 1 (College Composition) English 350/355 (Basic Reading/Writing Skills) 

English 62 (Analytical Reading) other; please name 

English 2 (Intro, to Literature) 
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Thank you for your help! 



End-of-Semester Anonymous Questionnaire 



To make sure classes at Solano are the best possible, we would like information about this 
English 1 /Political Science 1 Coordinated Studies class. Please help us by filling out this 
questionnaire. Your instructors will return it to the college Institutional Researcher. They will 
not read this form but will receive a typed report of students’ comments after the semester ends. 
Please do not put vour name on this form 

1 . Name at least 3 of the most important things you learned in this coordinated studies English 
1 /Political Science 1 class: 



2. Name at least 3 advantages of taking English 1 & Political Science 1 together in coordinated 
studies: 



3. Name at least 3 disadvantages of taking English 1 & Political Science 1 together in 
coordinated studies: 



4. Please estimate how many times per week you did the following for this English 1 Political 
Science 1 class: 

talked to your instructor in class 

talked to your instructor outside of class 

experienced the instructor talking to you individually in class 

talked in class with other students about class material 

talked outside of class with other students from the class about class material 

asked questions or contributed ideas in class 

worked in groups in class with other students 

worked or studied in groups on class material outside of olass 

5. Have the reading and writing skills you learned in your English 1 Political Science 1 
coordinated studies class this semester helped you in other classes you are taking? 

yes no 

If yes, please give an example of how these skills have helped you in another class: 



4 QDlease continue on the back... 



End-of-Semester Anonymous Questionnaire 



To make sure classes at Solano are the best possible, we would like information about this 
Political Science 1 class. Please help us by filling out this questionnaire. Your instructor will 
return it to the college Institutional Researcher. S/he will not read this form but will receive a 
typed report of students' comments after the semester ends. Please do not put vour name on 
this form . 

1 . Name at least 3 of the most important things you feel you learned in this Political Science 1 
class: 



2. Please estimate how many times per week you did the following for this Political Science 
1 class: 

talked to your instructor in class 

talked to your instructor outside of class 

experienced the instructor talking to you individually in class 

talked in class with other students about class material 

— talked outside of class with other students from the class about class material 
asked questions or contributed ideas in class 

worked in groups in class with other students 

worked or studied in groups on class material outside of class 

3. Are you completing any English classes this semester? yes no (if no, skip to #5) 

If yes, which one(s) 

— English 4 (Critical Thinking) English 350/355 (Basic Reading/Writing Skills) 

— English 1 (College Composition English 370 (English Fundamentals) 

English 62 (Analytical Reading) other; please name 

English 2 (Intro, to Literature) 

4. If you are taking English this semester, do you think the reading and writing skills you 

learned in your English class have helped you in other classes? yes no 

If yes, please give an example of how these skills have helped you in another class: 



5. Please make at least 3 suggestions for improving this class: 



6. Please list at least 3 features of this class that you think we should keep the same in future: 



7. Please add, on the back of this form, any other comments about this class and/or your 
experience in it. 

e/ccf 94 



Thank you very much for your help! 



[Sent to Experimental Group] 



FOLLOW-UP SURVEY 

During the semester of 199_ you were enrolled in a 

linked course which paired And English 1. 

We are trying to do a follow up survey of students who have 
taken these courses, and we would appreciate your input. Please 
answer the questions below and mail this survey in the envelope 
provided. Thank you for your participation! 

SURVEY QUESTIONS : 

1. Are you enrolled in college now? (Circle response) yes no 

a. At Solano? Yes No 

b. At another institution? 

2. If you are not enrolled in any institution, do you plan to 
return to school? 

Yes / When? No 



What we are trying to determine through our survey is whether 
students' experiences in certain kinds of courses have made a 
difference in their experiences in subsequent classes they take. 
So, the next few questions are aimed at having you compare what 
you experienced during one semester with what you experienced 
during other semesters. 

3. Describe any ways in which the skills and/or information that 
you learned in your linked class have helped you in. . . . 

Other English courses? 



Other History, Social Science, Biology, or Political Science 
classes you may have taken? 



4. a. In your English 1 class, did you discuss ideas and 
assignments outside of class with your classmates? 

Yes No 

Comments: 




4 ? 



4.b. Since taking your linked class, have you discussed ideas and 
assignments outside of class with your classmates more than you 
had before taking a linked class? 

Yes No 



Comments: 



5. A. How comfortable were you expressing your ideas and asking 
questions in your linked class? (Circle response.) 

Not at all not very somewhat very 

Comments : 



B. How comfortable were you in other courses expressing your 
ideas and asking questions? 

Not at all not very somewhat very 

Comments : 



6. Did your experience in U.S. History/ Biology/ Minority Group 
Relations/ Western Civilization/ Political Science stimulate your 
interest in 

A. Reading more about the subject? Yes No 

B. Taking more courses in this subject area? Yes No 



7. Did your experience in your English 1 course stimulate your 
interest in . . . 

A. The subject matter? Yes no 

B. Taking more courses in English? Yes no 



Code No. 



Control : 



48 



o 

ERIC 



[Sent to Control Group] 



FOLLOW-UP SURVEY 

During the semester of 199_ you were enrolled in an 

English 1 course and a course. 

We are trying to do a follow up survey of students who have 
taken these courses, and we would appreciate your input. Please 
answer the questions below and mail this survey in the envelope 
provided. Thank you for your participation! 

SURVEY QUESTIONS: 

1. Are you enrolled in college now? (Circle response) yes no 

a. At Solano? Yes No 

b. At another institution? 

2. If you are not enrolled in any institution, do you plan to 
return to school? 

Yes / When? No 



3. a. Describe any ways in which the skills and/or information 
that you learned in your English 1 course have helped you in any 
courses that you have taken. 



3.b. Describe any ways in which the skills and/or information 

that you learned in your course have helped you 

in any other courses you have taken. 



4. a. In your English 1 class, did you discuss ideas and 
assignments outside of class with other students from your 
English 1 class? 

Yes No 



Comments : 



4.b. Did you discuss ideas and assignments outside of cla'ss with 

other students from your class? 

Yes 



Comments : 



No 



4.c. In the courses that you have taken since your English 1 and 

, did you or do you now discuss ideas and 

assignments outside of class with other students? (Circle 
response.) Yes No 

Comments : 



5. a. How comfortable were you expressing your ideas and asking 
questions in your English 1 course? (Circle response.) 

Not at all not very somewhat very 

Comments : 



b. How comfortable were you expressing your ideas and asking 

questions in your course? 

Not at all not very somewhat very 

Comments : 



c. After completing your courses in English 1 and how 

comfortable have you been expressing your ideas in other courses 
that you have taken? 

Not at all not very somewhat very 

Comments : 



6. Did your experience in U.S. History/ Biology/ Minority Group 
Relations/ Western Civilization/ Political Science stimulate your 
interest in 

A. Reading more about the subject? Yes No 

B. Taking more courses in this subject area? Yes No 



7. Did your experience in your 
interest in. . . 

A. The subject matter? 

B. Taking more courses in 



English 1 course stimulate your 



Yes 

English? 



no 



Code No. 



Control : 



Yes 



no 



APPENDIX D: 



FIPSE PROJECT EVALUATION ACTIVITIES SUMMARY 



l 




Fall 1992-Spring 1995 



Z 

w 



-p 

u 

0) 

*o 

0 

u 

0* 

w 

(A 

04 



u 

0 



(A 

0) 

*rl 

-p 

•p 

> 

•P 

o 

< 

d 

o 

•rl 

-P 

cd 

d 

H 

cd 

> 

W 







IA 


U) 








42 


X 


<D 


P 




X 


X 


u 


o 


Ul 


P 


IA 


u 


o 


id 


id 


U) X 


O X 


P 


fd 


fd 


d) 


d) 


fd u 


X o 


P 


d) 


d) 






*— i fd 


fd 


O 








• 


U d) 


in d) 


X 


• 


Ul 


£ 


£ 




• 




£ 


p 






m 


H 


CM 




X 


o 


O 


i 


1 


1 


o 




in 


CM 


CM 






in 


<D 



P 

o 



vO 



d) 

0) 



E-* 



E-* 

U 

< 



Q) -P 
P P 
CnU 
Q) 

Q 



* 

P Ul 
o Ul 
id 

VO H 
rP U 
I 

^ *4 

0) o 

0) I 
S 4-> 

+ O 



l-l 

O 



l-l vO 
O fH 



+ 




+ 


+ 




Ul 




Ul 


U) 




U) 




Ul 


Ul 




fd 




fd 


fd 




p 


Ul 


«— 1 Ul 


rp 


Ul 


u 


Ul 


u tA 


u 


Ul 


id 


fd 




fd 


TJ 


p 


TJ rH 


TJ 


p 


d> 


u 


0) u 


0) 


u 


4J 




4-1 


4-1 




Id 


p 


fd p 


fd 


p 


p 


0 


P 0 


p 


0 


cn P 


cn p 


cn p 


d) 


4-> 


d) 4-> 


d) 


4-> 


4-> 


C 


4-1 c 


4-1 


G 


C 


0 


C 0 


c 


O 


M 


u 


H U 


H 


U 



l-l 

0) 4-> 
3 Ul 
O 0) 
P* E-» 

O' O' 
C C 
•P *P 
TJ TJ 
(d (d 
d) 0) 

o: pc 



>1 

0) 

> 

p 

0 

ui 

4-> 

c 

0) 

TJ 

P 

4-> 

Ul 



4-> 

*•-1 

C 

D 

£ 

p 

i— i 

p 

u 

•H 

P 

P 

P 

u 

>1 

id 

(A 

(A 

W 



4-> 

•P 

C 

P 

£ 

p 

i— i 

p 

o 

•p 

l-l 

P 

p 

L> 



CM 


CM P 
O 




4-> 


P 


P 


cm in 


Ul 


O 


o ui 


rp 


fd 




U) 




r— 1 Ul 


rH 


rP fd 


rP ^ 


Ul 




rp 


d) 


p fd 


X 


^ u 


^ d) 


d) rH 


d) 


d) 1 


d) S 


4-» O 


d) 


d) G 


d) 


44 




S M 


S + 


< 



CP 

c 

••-I 

4-> 

••-I 

P 

3: 

c 

o 

•p 

4-> 

(A 

Q) 

P 

o» 

>i 

rd 

(A 

(A 

W 



c 

o 

CP 

c 

•P 

£ 

i-i 

o 

z 



>1 

id 

tA 

Ul 

W 

0) 

P 

O 

4-1 

0) 

CQ 



i-l 

O 



vO 



Q) 

0) 



P 

O 



0) 

0) 

£ 



(A P 
>1 O 
fd 

tA VO 
(A 



>i 

4-1 

r— I 

P 

U 

rd 

4-1 

QJ 

X 

4-1 

4-1 

O 

tA 

Q) 

l-l 

•P 

tA 

Q) 

TJ 

Q) 

£ 

4-> 

C 

o 



CL) 
S-l 
* O 
l-l 4-1 
Q) Q) 
-P XL 
tA 

Q) tA 

a e 



o 

■P 
tA 
Q) tA 



i-l 

O 

4-1 



0) 

XL S-l 



d) 


cn 




0 ■ 




c 


Ul 


3 P 


P d) 


•p 


d) 


CL) 


d) d) 


TJ 


4-> 


E 4-> 


4-1 S 


C 


fd 


id ui 


44 


CL) 


TJ 


d) d) 


fd + 


0< 




4-> £ 




CL) 


Ul 


d) 


Ul ro 


TJ 


Ul 


>i ui 


>1 




id 


4-> 


fd P 


■«» 


i-4 


r4 cn 


TJ O 




O 


o c 




•— 1 


1 


L) P 



r- cm 
o 

4-1 0) 
0) 

rsi £ 



td 

£ 

i-l 

o 

4-4 

c 

•P 

S-l 

o 

CP 

G 

•rl 

4-> 

0) 

0) 

£ 



c id 

•P 4-1 

-p -p 
Q) Q) 
Ul Ul 



TJ 
C 
id 

tA 
4-> 

04 

Q) 

U 
C 

o 
u 

£ 

p 

r— I 

P 

u 

*r4 
P 4-1 
p 

a tA 
O CP 
G 

0) -P 
C TJ 

•r4 fd 

a Q> 

P P 
d> 

4-> TJ 
Q) G 
Q fd 



Ul 

tA 

fd 



G 

•P 

Q) 

G 

O 

TJ . 
tA 

tA 4 J 
•P *P 

0) 

> e 



o 

p 



4-i U id 4-1 U fd 



(A 



(A 



O 

•P 

P 

XL 

P 

PC 

CP 4-> 

C G 



•P 
P 

o 
o 

Ul r— I 

0) 

>1 > 
fd Q) 
tA Q 
tA 
W 



>1 

fd 

to 

tA 

w 

tA 

tA 

fd 

r— I 

u 

I 

c 



V0 



CP 

G 

•P 

P 

O 

o 

ui 

iZ 

CP 

c 

•p 

e 

p 

o 

z 

>1 

fd 

tA 

tA 

w 



G 

fd 

o 

tA 

d> 

•P 

-P 

•P 

> 

•P 

-P 

U 

fd 

CP 

c 

•P 

3 

O 



o 

4-1 • 

0) § 
x o) 

6-* 4-1 



tA 

*P 

-P 



tA 3 

0)3 

52 

<D 

S J= 

0) jj 

4-> 

c u 

5 0 
° i 

^ -P 
TJ^ 

s & 

10 C 

w 2 

a o 

o * 
2 o) 

•C 

* 0) 

5 



g 


£ 


u ui 


p 


p 


+ 3 




+ 3 


d) 


5 


fd 


fd 


J3 


d) 


Ul Ul 


U) 


to 




d) 


d) 


TJ r4 


d) 


X 


P TJ 


P 


TJ 


d) 


£p 


6p 


d> u 


£p 


x> 


O d) + 


0 


d) + 


X 






4J 




•p 


4-» 4-> 


4-1 


4-> 


4-> 


>1 


>i 


fd rp 




d) 


U fd ui 


O 


rd ui 




4J 


4J 


P 0 


-P 




3 P d> 


0 


p d) 


0 


rp 


i-4 


cn p 


H 


d) 


p cn ui 


p 


O' ui 


4-> 


2J 


2J 


d) 4-> 


G 


G 


4-1 d) ui 


-p 


d) ui 




u 


u 


4-> c 


U 


O 


Ul 4-1 Id 


Ul 


4-> id 


TJ 


fd 


fd 


G 0 


fd 


TJ 


G G «H 


c 


G »P 


d) 


Ip 


(p 


H U 


(p 


d) 


H H U 


H 


H U 


4-> 

O 



tA 

fd 



c 
o 
> V 

tA 
•p 



0) 

TJ 





G 


CL) 




D 


£ 






• P d> 




£ 


4-> > 




O 


P 


CL) 


rp 


Ul 3 


rp 


2 


Ul ui 


O 


O 


id 


T3 


•P 


rp d) 


d) 


P Ul 


L> X 


X 


p 


4-> 


o 


O id 


rp 


Ul 


u ui 


fd *w 




Ul 


4-1 


cn 


cn w 


0 


c 


G 


4J Ul 


-P 


•P P 


P 


c 


C 0 


d) 0 


c 


G 44 


X 0 


id 


id 


4-1 X 


rp 


rp 




0. 


IX 


Ul 


, 


. 


X P 


CO 


(T* 


B 0 



>1 £ 
0) O 
> P 
P 4-1 

P 

w tA 
m ^ 



P ^ 

S 1/1 

p> . 
tA *-* 
*P ^ 
c nJ 

I? 

fd *p 

O t5 

+J 0) 

So 
» u 

>1 ^ 
d) o 

4-1 

t ® 

P rP 

9i 

rG -p 

-P tA 

0) c 



tA 

p 

rp 

o, 



x: 

3 

Q) 

TJ 

•P 

U 

rti 

Xi 



o 

04 

(A 

0) 

p 

0) 

JP 



tA rP 

d) -P 
— 3 

tA 

U) tA 

fd u 

O 

U 4J 

. U 



vD 


vO 


T 

Ul 


vO 


fd 


P> X 
G 


-P X 
G 


4-r 

cd 

O X 


M 

4-> 

G 


44 


44 


Ul 


44 


G 


0 c 


0 c 

t« i T \ rri 


rP W 
rn 


O 



u 



Q) • • 

X tA tA 
4-> CP CP 
G G 
4-1 -P -P 
O T3 T3 
fd id 
2 | d) Q) 
.OJ p p 
p 

-P 4-1 

Ul o 



p 

p 

4-> 

tA 

G . 

•P T3 



Q> 

+ 0) TJ 

x: tj 

tA 4-1 Id 
tA 

id - Q> 
•H IA XI 
U tA 

cd C 

'S Id 

0) U u 

2^ 
o<0 
0) t .. 
X» tj X> 

c 2 

• P o 

d> 4-1 
X .p 
4-> 



P 

o 

U) 4-1 



U Ul U 

tA id P 

li rl P 
O O 
4-> 

u 4-4 
P O 
P 

4-> 4-> 



tA 

C 

•P 

d) 

X 



d) 

X 

4-> 

c 

o 



Ul 

4-> 

04 

<D 

O 

c 

o 

o 



c 
0 
•p 
Ul 
Ul 
p 
L> 

Ul 

•P I-H 
CP TJ (d 
G -P 
•P P P 
T3 O 



C *H 4J 
Q) m id 

o<“ e 

cu 5 

'O O d) 



>i 

a - 

> 



tA 

P 4J 
P C 
>, 0) dJ 
ff4-» Ul 
(A 
(A 

d> m 
0) 0 
•c G 



id Q} 
£ p 
04 



o 



o 

•P 

S' 

4-> 

G 4^ 
dJ 4J 
U) O 
d) CQ 
P 
04 



P tA 

c 0 ^ 

5 44 Id 

p 



d) o 
G 

0 0) 

i x 

i +J 

1/1 c 
>1 c , 



O 

TJ 



X 

X ° Jjj 
4j 5 Id 
m dJ 
44 4-> 

O 44 

o 



Ul 

c 

0 

•p 



c 

o 

•p 

4-1 

o 

p 

p 

4-» 

Ul 

c 

•p 

o 

4-> 



d> 

4-> * 



P 

O 

3 

P 

O 



4-» 


O 


d> 


fd 


Ul 


> 


p 


•P 


c 


d) 


d) 


p 


M 


TJ 


X 


d> 






4-> 


4J 




Ul 




id 


• 


>i4-> 


g 


Ul 


id 


G 




4-> 


TJ 


X 


Ul 


G 






•P 


d) 


d) 


•» 




TJ 


X 


E 


Ul 


O 




id 


•p 


4-> 


■H 


d) 


X 


Ul 


* 


4-> 





PI 



CVJ 

LTD 



er|c 



T3 

Q) 

P 

fd 

U 

Cn 

Q) 

P 

c 

P 

Q) 

x: 

P 

x: 

P 

O 

xa 

c 

P 

0 ) 

a 

-H 

P 



P 

a 

3 

0 

a 

d 

0 ) 

a 

fd 

w 

0 ) 

xi 

p 

p 

o 



XJ 

C 

fd 

0 ) 

a 

•H 

P 

a 

«d 

w 

0 ) 

X3 

P 

P 

«d 

0 ) 

c 

o 

XJ 

0 ) 

xa 

o 

p 

XJ 

0 ) 

0 ) 

c 

tn 

0 ) 

P 

p 1 

P 1 

P < 
U ' 
fd 1 



P O 
fd P 
p P 
0 ) C 

c o 
Q) o 

o> 

XJ 

c c 

m fd 



0 ) 

> 

p 

3 

W 

0 ) 

p 

0 ) 

rH 

cu 

e 

o 

o 



0 ) 

in 

fd 



M-l 

O 

in 

0) 

0) 

3 

0) 

0) 

p 

XJ 

p • 
in 

p in 
in fd 

P rH 

p u 

4-4 

4-4 

c o 

P 

x: in 
P M 
P 0) 

3 0) 
3 

rH 

0) 

>i 0) 

fd p 

in x: 
in p 
w 

p 
xj m 
c fd 

fd rH 

p c 

••H *r| 

c x: 
d p 

P 

e 5 

3 

rH (N 

3 

U >i 
•<h fd 

p in 
p in 

p ro 

U * 

XJ 
- C 

rH nJ 

cu P 
P$ P 
Q C 
D 

P e 
3 

>1 rH 



Q) 

> 

P 

3 
W 

Q> 

P 
0 ) - 
rH CM 

04 

e cu 
o o$ 
U Q 



C 0) 
P C 

0>° 
o c 

C P 

0 ) x: 
p p 

Q) P 
4-1 j 
4-1 

•H Xf 
XJ d) 

rH 

O 3 
C XJ 
0) 

in jz 
P O 

0) w 
P Q} 

Q) x» 
x: 

P o 
in 

O rH 

in fd 
X T3 

Q) rH 

0) 3 

3 o 
jz 
Q) tn 
c 

o in 
0) 

c tn 

•H w 

x: td 

P rH 
-rH O 

rH 

in o 
p p 
in 4 J 
Q) c 
p o 

a, ° 
VC TJ 
o c 

fd 

0) 

P XJ 
3 Q) 
XJ P 
0) fd 

x: p 
o cn 
tn a) 

O £ 

P -rH 



c in 
tn q) 



fd w 

rH 

O Q) 



O 

P 

P 

c 

o 

o 

T3 

C 

fd 



x: 

p 

o 

m 



CN 



CnxJ 
C O 
•H O 

p x: 

•H .rH 
P rH 

3 0 ) 

5 - 1 

tn 

in o 
® w 



in 



<d 



in « 

fd - 1 

u 

i ^ 

c xj 

•rH 

T3 
T3 C 
C fd 
fd 



in 
P 

-rH 

g* 



o> 

c 

p 



p 

0 ) 

Cn 

p 



c 

fd 



3 ° 

u ^ 
P'2 

p 

3 

° o> 

P c 

in -rH 

o c 

04 P 
_ fd 
T3 m 

S- 



0 ) 

a £ 

04^ 



. u 

p (U 

o sz 



Q) 0 ) 

P E Q> 
3 P XJ 
XJ P P 
0 ) 

£H C 

o fd ■ 



W IU -rj 

in p 

n fr® 



O 

P 

XJ 

0 ) 

0 ) 

c 



in 

fd 



xj p u 
Q) o 2 

p 4_, x; 

m p 

P Q) o 

cn e 
a > ^ x: 
p p o 
c fd 

-rH 4 h Q) 

o 



TD 0) 
0 ) U 

p «d 

fd th 
P TJ 
Cn fd 
d) 



p 
P o 
C 

3 XC 
O 0 ) 
| 0 ) 
S 5 



p 

c 

•H 

a) 

x: 



O d) d) c 

CX O4O 
P >1-H 
P , P P 
fd u fd 

x: o u 3 

p P fd rH 

O c > 

in g e w 

- 0 'n w 
S *P 2 . w 

„ M 
fd tn 
tn c 
in o q} 

d) ,rH x: 
p p 
0 ) tn 

c x: d) c 
rH P 3 —I 



p 



- 0§, S 

O XJ p E 
o 

in 

d) D 



P T3 
**H C 



cnx; 
c p 

fd 

P T3 
d) 



d) TJ 
C 

tn <D 

•rH P 

P 

Q) P 
p _ 
in d) o 

d) x: xj 

* ^ a 

4 J P 
rH 04 

o x: o 

p xJ 



>1 

<d 

in 

tn 

d) 

d) 

x: 

p 

in 

in 

fd 

O 4 



•H 

m 

P 

a 

d) 

T3 

3 

p 

in 



d> 

x: 

p 

p 

dJ 

p 

p 



c 

d) 

p 

p 

P 

P 

3 

d) 

xa 

o 

p 

tn 

T3 

d> 

d) 

c 

c 

o 

•rH 



_ Q) S 
fd xj 3 

c V c 

.258 

. p 0 
XJ x 
in 4 J Q} 

d) P > 

^ o£ 

Q} P 
p — 

fd tn c 
O 4 TI o 
dJ d) p 
p a) in 
O 4 c in 
_ o 
O c u 
XJ o w 
rn P P 
2 p T3 

58' 



in 

p 

g 

So.g’g 

w x: p tn 
t-f TJ 
rd w 
d> tn 
' . p ~ 

Cn. 



o 
p 

c u 
5 o fd 

S 2 10 

P m ® 
> “ P 
^ fd 



p x: 
tn u 

di fd 
3 d) 
&P 

tn p 
cn 
d) 

0) 
x: 

H 



>i ftj C 4 

° fd c § 

p s 8 

O' d) T3 
c c 
cn m 1 



d> 

•O 



c 

id 

o 



in 
p 
cn 

Q> m 
jz in 

P XJ 
C P 



tn 

>1 

fd 

in 

in 

d) 

d) 

x: 

p 

d) 

p 

fd 

04 

d) 

p 

04 

o 

p 

c 

fd 

in 

3 

w 

T3 

C 

fd 

xa 

o 

« 

p 

o 

4h 

d) 

e 

-rH 

P 

TJ 

fd 

d) 



in 

fd 

TJ 



x: 

p 

p 

3 

0 4 

3 

p 

d) 

tn 

d) 

xa 

o 

p 

T3 

d) 

d) 

C 

in 

c 

o 

p 

in 

tn 

d> 

in 

Cn 

c 

p • 
p tn 

O 4J 

o % 
in m 

nr, O 



d) 

x: 

p 



o> 

c 

p 

p 

o 

o 

in 

T3 

c 

fd 

Cn 

c 

p 

E 

P 

o 

c . 
c 

pa q) 

W 4 J 
04 Z 



d> 

2 Q> 

3 4J 

m-g 

in 

fd 4.4 



c 

o 

p 

•H 

d) 

x: 

p 



c 

p 
in ^ 

S| 

Q) ^5 
Q) d) 

v o 

CnXJ 

'■s 

si 

p 5! 
Cn c 

0 } tn 
p c 
p 

rH P 
rH O 

P o 

5 tn 

^'O 
P c 



fd 

04 



o 

p 

TD 

d) 

d) 

C 

d) 

tn 

d) 

x: 

p 



>, in 

>1 



p 

p 

3 

in 

fd 

3 

>1 

fd 

in 

tn 

d> 



•rH 

in t 3 
in <d 



E 
P 
O 

o c 

? I 
P c 

O sP 



P 

c 

p 

T3 

(U 

T3 

P 

> 



C 

, o 

•■o' 0 
0) 8 



fd 



fd 



in 



o w 

0 ^ 
tn ij 
T3 

- S 



fd 

in 

in 

d) 



in 
3 
o 

p 

p 

d) d) 
in in 
d> 

E -53 
d) 



0 


5^ 


in 


fd 




d> p 


4h 




p 


fd >1 

P p 


• ^ 


rH 


d) 

0 

c 

d> 

p 


w u 
^ 8 
5 


c 


3 Q> 


<D 


5 -c 


> 

C 


XJ JJ 
in ^ 


0 


4h 


0 


rH t— 1 
rH 


p 


fd 


0 


• 


4h 


*s “ 


in 

d) 


p 


XJ p 


p 


3 


in 


0 in 


d) 


in 


T3 


d) 




P ^ 
p XJ 


s 

p . 


H .5 


4h 


0 O 


P 


XJ 




in 


in 


in P 


c 


fd 0 


0 = 


rH 4H 



o o 
p in 

u -a * 

fi « 18 



p , 
o , 

d) 

E 

o 

in 



p 

d) 

Cn 



o 

p 



p 
<D 
4h 
4-1 
T3 O 
0) 

d) O 
C P 

in 'O 
p 
in 
d> 



d) 

tn " 

fd >* 
p 



w p 

g'o 



in q) 
tn 



in 

3 

O 

P 

P 

d) 

in 

cn 

>1 

d) 

> 

p 

3 

tn 





in tn 


tn O' 


4-) 


•rH 


o> 0 


in 


in 




p 


in qj 


c c 




E 


C ftJ 


a> 


p 


XJ 


0 


w Cn p 


p 


d) 


P 


•p *n 




0 


c 


P 


fd c 


E £ 


O 


O 


E 'O 


03 


P 


fd 


u 


P P 4J 


P ^ 


C 


c 


P nJ 


sz 


u 




3 


O > fd 


O 0 


P 




O 


£h 


3 


v 


P 


p x: 


z c 


w 


04 


2 ; c 




p 


in 


P 


POP 






3 


O 




p 


P 


in 


x > 


V 




O 






in 


in 


c 


d) c 0 


- c 


• 


P 


• d) 


• 


c 


0) 


p 


c p in 


in id 


vD 


0> 


r- p 


00 


M 


P 



no 

no 



no 



BEST COPY AVAILABLE 



U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and improvement (OERl) 
Educational Resources information Center (ERIC) 




NOTICE 

REPRODUCTION BASIS 



This document is covered by a signed “Reproduction Release 
(Blanket)” form (on file within the ERIC system), encompassing all 
or classes of documents from its source organization and, therefore, 
does not require a “Specific Document” Release form. 



This document is Federally-funded, or carries its own permission to 
reproduce, or is otherwise in the public domain and, therefore, may 
be reproduced by ERIC without a signed Reproduction Release 
form (either “Specific Document” or “Blanket”). 




